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ABSTRACT 

The results of research on the effectiveness of four 
alternative instructional media are reviewed: traditional classroom 
instruction, instructional radio, instructional television, programed 
instruction, and computer-assisted instruction (CAI) . It was 
concluded that studants learn effectively from all these media; 
relatively few studies indicate significant differences between media 
in effectiveness. Future research should focus on four areas: 1) 
determine if programed instruction and CAI can be shown to save 
instructional time over a longer period and with a higher percentage 
of students; 2) investigate long-term effects cf instructional 
technologies on students* motivations; 3) investigate the long-term 
effects of the individualization and privacy made possible by some of 
the technologies; and 4) future investigations should consider more 
imaginative uses of new techtiologies instead of using technology to 
provide a simulation of some traditional method. (RU> 
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THE EFFECTIVENESS OF ALTERATIVE INSTRUCTIONAL MEDIA: A SURVEY 



This survey provides an overview of research on the effectiveness 
of alternative instructional nedla. The media discussed are tra- 
ditional classroom instruction (TI) , Instructional radio (IR) , 
instructional television (ITV) , programmed Instruction (PI) , and 
computer-assisted instruction (CAI). The effectiveness of these 
media Is examined from a reasonably macroscopic point of view; 
the psychology of pupil-teacher Interaction or the 'content variables' 
of ITV, to take two examples, are at a micro-level not considered. 
Achievement test scores constitute the measure of effectiveness most 
frequently used In this survey though, where available, results con- 
cerning the affective impact of the various media of instruction are 
included. Achievement test data, la most cases, were collected only 
on an annual basis, so they reveal no fine-grained detail about the 
learning process. 

Since this survey is relatively brief and its scope broad, a few 
caveats are In order. First, where literature surveys are available, 
their resultd have been cited to the extent possible and, frequently, 
original sources remain unchecked. Second, available knowledge of the 
effectiveness of the various methods varies considerably; much more Is 
known about TI and ITV than about the others. For this reason a survey 
such as this is inherently spotty in Its conclusions. The third warning, 
related to the second. Is that many of the evaluations fall short of (or 
lack entirely) scientific standards of analysis and reporting. For this 
reason, it was sometinea necessary to attempt to cull conclusions from 
essentially Journalistic accounts of projects. Fourth, it should be 
noted that this survey is limited to instruction within a school setting. 
Finally, the survey excludes Infomation on costs. 

Before beginning the literature survey, we present an 'ideal' 
paradigm for measurenent of effectiveness and then discuss several less 
desirable alternatives that have actually been employed. In the surveys 
of die Individual methods where adequate prior surveys are unavailable. 
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results from a representative sample of Individual evaluations are 
discussed. Where adequate surveys are available, their conclusions are 
presented with a description of one or a few specific project evaluations. 
In addition t- a nuni>er of aedlui'i specific survejrs there exist several 
reviews — Allen [1960], Chu and Schram 11967], and Schram [to appear] 
— that cover more than one of the topics dealt with In this review. 
Our objective is to attempt to bring together the overall results for 
all the principal media; other of the reviews mentioned here and 
elsewhere in our review sonetlmes have more detailed and specific references 
to the literature in some particular area than we are able to provide. 
The media are discussed in appraxlmately the temporal order in which 
they were Introduced; Individuals with a particular interest in one 
meaium are urged to skip directly to the appropriate section. 
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I. ASSESSING THE IMPACT OF ALTERNATIVE INSTRUCTIONAL MEDIA 



An Ideal atudy of the cognitive effectiveness of alternative 
Inatructlonal nedla would relate a vector of output measures relevant 
to a subject matter » including criterion-referenced measures of 
achievanenty to the time pattern of instructional Inputs. This 
function would include as Independent variables factors not under 
the control of the school system so that, in its allocation of resources » 
the system could provide, to the extent desirable, different patterns 
of resource inputs to different categories of students. In order to 
assess the effects of different mixes of media and total amounts of 
time spent in learning a subject matter, we would need an experiment 
of vast magnitude; present survey methods are inadequate because of 
the current lack of stibstantial variation in methods of instruction. 
Since these methods are now virtually 100 percent TI, survey methods, 
as reported in the next section, can be xised to assess the effect of 
different types of TI. 

If it were to be possible to conduct an ideal experiment, the 
resulting function relating the educational system's outputs to its 
inputs would be of great value In efficient allocation of resources 
to and within sdiool systems. This Is primarily because the effect 
on output of more or less of any one input would be known as a function 
of the levels of all the Inputs. E^/en with much less ambitious 
experimentation it is possible to obtain some idea of how output varies 
with input througih simple multivariate regression models. For example, 
to assess the lapact of CAI drill and practice in arithmetic (assimilng 
CAI as an addition to and not a substitute for TI in arithmetic) , let 
us postulate a model of the follovlng form: 



where A^^ is arithmetic achievement at the beginning of year k , C^ 
is the nuiber of CAI sesslona the student has In year k, and b^ , 
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, and are parametera to be estimated. Some results of CAI 
and TI surveyed are basically models of this form, though seldom do 
the TI studies have data that are either longitudinal or on a student- 
by-student basis. While models of this general sprt, employing a 
variety of functional forms, give a quantitative estimate of hov output 
varies with a few inputs, they fall short of the ideal by being inadequate 
for examining the impact of mixes of instructional technologies over 
time. 

Still less informative are studies that examine whether 
supplementing TI with a technology or replacing it with a technology 
will yield achievement results that are significantly different from 
TI, because the magnitude of the effects, when they do exist, cannot 
be obtained in functional form. The vast majority of good evaluations 
of educational technologies are, however, of this general format The 
good studies provide controls by careful matching or randomizixsg and 
thus provide statistically valid results. Many more studies lack 
adequate controls or are in other ways flawed. That the results fre- 
quently indicate "no significant difference" is a valuable finding, 
not sufficiently used and appreciated in selecting a medium of instruction. 

Finally^ least satisfactory for purposes of assessing perfor- 
mance are projects whose evaluations are essentially journalistic. 
While much can be learned from good journalism, it is difficult to 
avoid feeling uneasy without supportive data, particularly if claims 
about substantial Improvements in performance are made. It should be 
stressed, however, that there Is no proved correlation between the 
effectiveness of a project and the sophf^tication with %ihich it is 
evaluated. For this reason journalistic accounts can provide valuable 
screening for more detailed examination of projects that show potential 
for widespread use. 



^his perhaps results from what the authors feel is an over- 
emphasis on a control vs. experimental group methodology relative to a 
methodology that seeks to model input-output relationships. To take 
one exaiq>le, Suchman [1967] paid almost no attention to the problem 
of ascertaining how the amount of effect Is related to the amount of 
stimulus » to use his terminology. 
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This paper reports, then, on evaluations of varying degrees of 
adequacy and attempts to draw some general conclusions about the 
relative effectiveness of alternative instructional media. Schramm 
11971] stressed the difficulties involved in making scientifically 
valid cross-media comparisons, and we share many of his reservations.^ 
Yet a number of reasonably dear patterns do emerge from the data and 
these are what we report. 



2 

Schranm also discussed how beat to design experiments to make 
these comparisons. The central problon is that the number of potentially 
relevant variables to be controlled* or orthogonally varied. Is so large 
that 'experiments' becooe substantial real^rld projects over which the 
experimenter may end up having little control. 



II. TRADITIONAL CLASSROOM INSTRUCTKW 



This section reviews the deteimlnants of a student's scholastic 
achievement In a traditional classroom setting. Much of the work 
reviewed uses multiple regression analysis to relate a student's 
achievement test scores to attributes of his school environment 
(Including the composition of tne student body) , his background and 
socioeconomic status » and his teachers. Many of the studies utilize 
the extensive data base provided by the Equality of Educational 
Opportunity (EEO) survey and first analyzed In Coleman, Campbell, Hobson, 
McPartland, Mood, Welnfeld, and York [1966]. Coleman et al. concluded 
that variation in scbool Inputs accounts for at best a very small 
fraction of the variation In student achievement; socloeconoBDlc 
variables, they concluded, are mucb more central. Other analyses, 
some of them also based on the EEO survey data, are sunmiarlzed later 
In this section and a number of theu find more evidence for the efficacy 
of the things schools provide t3xan Coleman et al. do. However, the 
findings are often Inconsistent and a recent thorough review of this 
literature (Averdi, Carroll, Donaldson, Klesllng, and Plncus, 1972] 
Hated as a basic conclusion: 

"Proposition 1 : Research has iK>t Identified a variant of 

the existing system that jLs consistently related to students ' 

educational outcomes /' 
In a recent reanalysls of much of the data used In the studies Averch, 
et al. surveyed, Jencks ec al. [1972] reached much the same conclusion: 
"tfe see no evidence that either school administrators or educational 
experts kncxr how to raise test scores. Certainly we do not know how 
to do so [p. 95]." 

The reason is not that no studies have found significant iiq>ut 
variables. Rather, Averch et al. [1972] state: "The literature 
contains numerous examples of educational practices that do seem to 
have significantly affected student outcomes. The problem is that 
other studies, similar In approach and method, find the same educa- 
tional practices to be ineffective; and we have no clear idea why 
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this discrepancy exists Ipp. x-xij." The present survey accepts »:his 
basic conclusion, with only minor reservations that are stated later. 

This section first presents a sunauary of 17 studies in a 
readily usable, tabular fonii;^ it then provides a brief verbal summary 
of some of the more significant Tendings. After the susmnary of studies 
based on survey data, this review examines more closely the literature 
on what Is perhaps the mst economrlcally significant variable under 
the control of a school system — class size or student-to-teacher 
ratio. The significance of this variable lies In the observation that 
Increasing the aggregate student-to-teacher ratio by either enlarging 
class size or reducing the number of periods the student spends with 
the classroom teacher constitutes the principal available way of 
substituting capital for labor In the educational system. It is thus 
important to ascertain what negative effects would need to be compen- 
sated for (or more than compensated for) by Introducing a technology. 
Some of the studies we discuss are based on experlnental , as well as 
survey data, and others report affective Impact. As indicated at the 
outset, the present survey does ^ deal with relatively micro-level 
variables relating, for exaaple, to the psychology of pupil-teacher 
Interaction.^ 
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Earlier surveys of parts of this literature appear In Guthrie 
[1970] and Katzman [1971] ; more iq>-to-date surveys are Included in 
Averch et al., [1972] and Anoerson and Greenberg [1972]. The studies 
reviewed in this literature were undertaken prlaiarlly in the United 
States; International data and cumptLTlmoaB are more difficult to find. 
An exception Is the Tntam«^^«n«^ study of Acuievement In Hathematics 
edited by Husen [1967], This study suggests that the pittem of results 
found In the United States la more generally ^pllcable. 
4 

For a review of ouch of this literature see Bosenshlne [1971]; 
he reviewed 51 studies of the relation between specific teacher behaviors 
and student achievement, and observed that studies of this sort have 
had a better history of finding sigaificant Influences on student 
achievement than have the studies of the effect -jf teacher personality 
and background variables that are the focus of toe present review. 
CUrlty of the instructor's preaentational style was one of the most 
liiQ>ortant variables he found. 
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Survey ^ata Studies of School Effectiveness 

/ 

The review of these stu^'lea Is suamarlzed In Table 1 which Is 
subdivided by variable. . .^eludes mainly variables under the 



Insert Table 1 about here 



control of school systems though Burkhead, Fox, and Holland [1967] and 
Coleman et al. [1966] both stressed the primary Importance of the 
socioeconomic variables and used step-wise regressions to enter these 
variables first. Due to the multlcolllnearlty problem^ between 
socioeconomic group and school variables, this procedure biases the 
regression results In these reports In the direction of concluding 
that school resource variation does little to predict achievement score 
variation. Many studies that followed the Colemat. report and used data 



Thft multlcolllnearlty problem arises because. In general, higher 
Income districts have more money to purchase higher quality resources . 
For example, Guthrie, Klelndorfer, Levin, and Stout [1971] examined the 
lelationship between socioeconomic status and quality of school resources 
and found a positive relationship on individual, school, and district 
bases. The effect of entering socioeconomic variables in the regression 
first, as did Coleman et al. [1966] and Burkhead [1967], is that the 

1 reduction in variarce attributable to socioeconomic status Includes a 

joint effect with school resources. When school resources are entered 

^ into the equation, their importance is diminished,, since only the unique 

contribution for school rei^ources is measured. Mayeske [1970] evaluated 
the unique contributions of eaCh set of Inputs and the joint contribution 
of the two sets using analysis of variance. He concluded that out of 
the total amount of variance accounted for when both background and 
school variables were in the equation, 12 percent were uniquely ideatified 
with backgicuud variables, 6 percent with school variables, and 82 
percent jointly. Clearly, with school resources entered second, back- 
ground accounte'j for 94 percent and school for 6 percent of the total 
variance. Stratification by social class [Benson, Sclmelze, Gustafson 

I and Lange, 1965; Guthrie, et al., 1971; Hanushek, 1970; Riesling, 1967; 

j Michelson, 1970] ia one possible method of dealing with these multi- 

l collinearity problems. In each o£ these studies there vwre some school 

I variables which were significant. 
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from it (such as Adelnan and Parti, unpublished, Bowles, 1970; Levin, 
1970; Michelson, 3970) do, however, show a significant relationship 
between various school resources and studenc achievonent. To the 
education administrator or policy maker, the existence or extent of 
the effect of socioeconomic variables is far less iin>ortant than a 
finding that school resources have a differential effect on children, 
depending on their background. Such an effect Is evident in results 
of Camoy [1971], Hanushek [1970], and Michelson [1970J, as well as in 
Coleman et al. [1966]. 

Coleman et al. based their results on the amount of variation 
explained by a group of variables after socioeconomic variables were 
entered in the regression. Four groups of variables were used: 
socioeconomic, teacher, school and student body variables. For ninth 
and twelfth graders, the teacher characteristics added 8 percent to 
the axplanatory power, or raised R by .08 in the equation for 
Southern blacks, .03 for Northern blacks, .022 for Southern whites, 
and .015 for Northern whites. These variations might have been higher 
if the teacher verbal score, which according to the report bears the 
highest relationship with student achievement, had been included in 
the group of teacher characteristics. The Importance of the above 
results Is that there is a differential Impact on achievement depending 
upon the student's race and geographic region. 

Hanushek [1970] used the EEC survey data for sixth graders in the 
Northeast and Great Lakes region and stratified by race. He used a 
multiplicative model, and the regression coefficients were output 
elasticities (a), that is, the percentage change in output for a 1 
percent change in input. For teacher experience and teacher score on 
a 30-point verbal test, the results differed for blacks and whites. For 
teacher score, a - .117 for lAltes, and .178 for blacks; for teacher 
experience, a - .02 for whites, and .045 for blacks. For both 
teacher variables, there was a higher Impact on the black achievement than 
on white achievement. If there were a correlation between race and 
socioeconomic group (with whites being from a higher socioeconomic 
group than blacks), these results would contrast with those of Camoy 
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[1971] where teacher experlem a had a greater Impact on high socioeconomic 
students than on low socioeconomic students. 

In the same paper Hanushek analyzed data for third graders in 
one school district in California. As opposed to the EEO surv::y data, 
where average teacher characteristics by school were applied to each stu- 
dent or to average student achievement. Hanushek was able to match students 
with their second- and third-grade teachers. The students were then 
stratified by ethnic background (with or without Spanish surname) and by 
the occupation of the head of the household (manual or nonmanual labor). 
There are only three groups since in his sample there were no Spanlsh- 
sumamed children from a home in which the head of the household had a 
nonmanual job. The teacher characteristics analyzed are teacher expe- 
rience, teacher verbal score (on a 100-polnt test), number of graduate 
units, teacher experience with socioeconomic class and number of years 
since teacher's most recent educational experience. Teacher experience 
and education were not significant In explalaf.og achievement for any of 
the groups, and there was no teacher characteristic which explains 
achievement of Spanlsh-sumamed children. This differs from his other 
result that school resources have a larger effect on minority children, 
perhaps because of the language difficulties of Spanlsh-sumamed 
students for whom EngUsh was a second language. Hanushek [1972] 
pr-vldes an extensive discussion of these results. 

The studies Just discussed provide a sample of the type of 
analysis that the studies summarized In Table 1 represent. What does 
emerge from those studies, and from the tabular sunnary. is a striking 
lack of uniformity concerning the significance cf various variables. 
Further, more targeted research will be required to ascertain more 
exactly the nature of the conditions that make significant a particular 
factor of instruction. 

Table 1 Included only studies at the elementary and secondary 
level; Dubin and Taveggia [1968] surveyed the results of 74 studies 
that compared various teaching methods at the higher education level. 
In most of the studies students were randomly assigned to one of two 
methods of teaching; the results do not gJ.ve. then, regression 
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coefficients that could be used to examine the magnitude of the effect 
on output of various levels of change in input. Though individual 
studies may have concluded one method of teaching superior to another, 
Dubin and Taveggia concluded from all of the studies taken together 
that there was no evidence for the superior effectiveness of one 
teaching method over another at the college level. The metliods included 
in their survey included lecture sections, discussion, and supervised 
and unsupervised independent study. 

A recent regression analysis of the determinants of economics 
achievement, based on extensive survey c'.ata, is perhaps the best study 
to date of input effect^.venaas at the university level. Attiyeh and 
Lumsden [1972] sumcarized this long term study in a recent paper; 
more detailed analyses are referred to there. The output measure used 
was the score of the student at the end of the year on an objective 
examination stressing the student's ability to apply fundamental 
economic principles to the solution of real-life situations or prob- 
lems. The independent variables included pretest score, student back- 
ground variables (age, sex, year at university, general aptitude, 
attitudes, and field of specialization), faculty characteristics (age, 
experience and rank of lecturers and tutors), and course characteristics 
(class size, hours devoted to microeconomics, hours devoted to macro- 
economics, and course materials in both lectures and tutorials). The 
student's attitudes toward the course and lecturer were not s.'gnlflcantly 
related to posttest score^ but the student's opinion of the "us ".fulness" 
of economics was. Of the controllable variables tutorial size w&t 
significant while lecture size (with a range of 30 to 400 stur ents) was 
insignificant; rank, age, and years of experience were slgnif Leant for 
lecturers aud insignificant for tutors. The number of class hours was 
significant. 



In another study of student evaluations Rodin and Rodin [197?] 
found tiiat "Students rate most highly instructors from whan they learn 
least. These findings of the invalidity of student ratings are not 
supported In a revlw paper by Cottln, Greenough, and Manges [1971]. 
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Effects of Class Size 

As class size Is perhaps the not t economically significant 
variable In TI, we will deal with it ii slightly greater detail at 
this point. In Table 1 the variables "class size" and "student to 
teacher ratio" were seen to be insignificant in all but 2 of the 
regression studies that reported using these variables. This subsection 
discusses a number of additional studiea of the effects of class size, 
including several experimental studies. 

A frequently cited review of the early literature on the effect 
of class size is Blake 11954], whidi is summarized in Sitkei [1968] 
and Vamer [1968] . Blake summarized 85 studies on the effects of class 
size in piblic elementary and secondary schools. Of these, 35 favored 
smaller classes, 32 were inconclusive, and 18 favored larger classes. 
When stricter requirements were imposed on statistical procedures , 16 
studies favored smaller classes, 3 were inconclusive, and 3 favored 
larger classes . An additional survey of early literature on class 
8l.^e may be found In Fleming [1959], This is the background for more 
rectnt studies which in some cases provided regression coefficients 
that .-urn be used to eat. mate the change in achievement to be erpected 
with given changes in class size. In the following discussion, results 
are also reported in some ^ixperlments and surveys where regression 
coefficients or elasticitiec are not available, though some of these 
results can give an Impresaioix of the size of the effects, several 
studies of tho effects of class size are first summarized in the text; 
following that Is a table sumurLzlng these and other results reported 
subsequently to Llake's IS54 surve/. 

Ftymier [19C4] surveyed 12 Fiorida school districts and then 
selected all classes with more than 3S students and all with less than 
30 students In the first grade. There were a total of 201 students in 
the larger classes and 119 In the smaller ones. The laxjjer classes 
scored significantly hightr at the beginning of the year on the Metro- 
politan Readiness Test. At the end of the school year (May) the students 
were given the WilUame Priaary Reading Teat irtth the result tf «t 
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students In the smaller class scored better at a significance level of 
.001. The difference in grade placement was, however, slight; for 
the small classes it was 1.75 and, for the large, 1.62. Though there 
were no controls for the many other possible factors, physical handicaps 
and teacher differences were checked and were not found significantly 
different in the two groups. 

In another study at the primary (K-3) level, Balow [1969] found 
small dassfts superior to large ones; the difference was statistically 
significant at the .01 level, but not large in absolute terms. The 
classes were assigned to conditions randomly. A mora detailed analysis 
showed that the difference was due to learning among boys. In the 
subsequent two years the students who had been in the small classes con- 
tinued to gain more iJian those who were assigned to small classes after 
having been in a larjje class for the first grade; in the second year, 
however, the difference was not statistically significant. Balow 's 
interpretation of the results was that small classes are Important the 
first year; after that the difference la not significant. 

In Sweden, M«.rklund [1963] found that in a large sample o£ 
sixth-grade classes,, those with 26 to 30 students learned the most . 
After that came the 16-20 group (smallest) and the 31-35 group (largest). 
Among classes that had slxth-grada students combined with other grades, 
the smallest dasseu were favored. In f:-3mparlsons among students 
divided into groupc. according to socioerouoalc status, IQ, homogeneity, 
etc., 22 comparisons favored ismaller classes, 37 favored larger, and 
222 were not significantly olfferent. 

Johnson an4 Scrlven [1967] used data from the New York Quality 
Measurement Program to examine tue effects of class size. From the 
total sample only those classes within 0.3 of the mean of the class 
In terms of grade level on die pretest were examined. English and 
mathematics classes In grades 7 and 8 were the subjects of the study. 
Random sanq)llng was used to derive equal nunbers in cells for an 
analysis of variance. The results favored larger classes in 10 out 
of 16 comparisons. The small classes did relatively better for the 
seventh-grade students and for studAnts abov» the mean on their pretest 
scores. 
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Table 2 sumnarlzes a number of studies on the cognitive effects 



Insert Table 2 about here 



of class size that were undertaken since the time of Blake's 1954 
survey. Not included in Table 2 are the extensive results of the 
International survey reported in Husen ]1967] ; volume II of that 
study (pp. 79-85) reports on numerous comparisons of different sized 
mathematics classes. The results were usually no significant 
difference and, where significant differences were found, they were 
more likely to be for older students. 

While the relationship between class size and achievement is gen- 
erally weak, some researchers believe that the interpersonal aspects 
of the classroom suffer with Increased class size. Olson [1971] found 
an advantage for smaller classes In terms of individualism, interpersonal 
regard, group activity, and creativity. This survey obtained data from 
almost 10,000 classrooms at the elementary level and 8,600 at the sec- 
ondary level. Smaller classes were favored at all levels. Using the 
same sample, Vincent [1968] found Inconsistent relationships between 
class size and achievement. 

In a smaller study. Cannon [1966] reported that In two kindergarten 
classes (one with 34-39 students, the other with 23-28 students), the 
smaller class was favored In terms of fewer aggressive acts , better peer 
relationships, more and better child-teacher contacts, more creative 
activities, and better feelings on the part of the teacher. The 
differences were not large, however. 

Thus at the elementary level the quality of interaction appears 
to be Inversely related to class size. At the secondary level the 
matter Is not so clear. Olson [1971] r^orted that observational data 
supported leas attractive styles of Interaction as class size Increases. 
Anderson, Bedford, Clark, and Schipper [1963], Ed. W. Clark High School 
[1968] , and VilUma and Koeladie [1967] reported no difference in 
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student-reported attitudes tovard the class. The finding for secondary 
schools appears to hold at the coimunlty college level as wall. There 
were no significant differences in the studies examined except that 
Hopper and Keller [1966] Indicated that students prefer the larger 
classes. For students who do not particularly care to participate lu 
discussion, the large class can be superior. 



Conclusion 



In concluding tills survey on the effectiveness of traditional 
classroom Instruction, It seems reasonable to agree with Averch et al. 
[1972] that few variables consistently make a difference in student 
performance. Exceptions to this general conclusion would be that teacher 
verbal abiUty appeals Important in a high fraction of the Instances 
examined, and that small classes seem to Improve the cognitive and 
affective performance of young children.^ This conclusion does not, 
however. Imply that schools mdce no difference in the cognitive 
development of their students; on the contrary, school attendance is 
clearly Important In promoting academic achievement though few studies 



In light of this finding it is perhaps ironic that national 
average pupil- to-teacher ratios are substantially higher at the 
elementary level (24.8:1) than at the secondary level (20.0:1). These 
figures are for 1969 and are from the United States Office of Education 
[1970, p. 59]. Stevenson [1923, pp. 122-125] noted this anamolous 
situation a half century ago. He estimated class sizes then to average 
38 at the elementary level and 25 at the senior high school level; 
his resaarcfa concluded that the only noticeable advantages for small 
classes were at the elemmitaxy level, particularly for dull pupils. 
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seem to have examined this Issue.^ It remains to be seen that varia- 
tions in school inputs are consistently related to variations In school 



outputs , 



8 

rio*/i referred to a study undertaken by Green et al. 

I 1964] on the effects of closing the schools In Prince Edward County, 
Virginia as a result of court-ordered desegregation. Students who 
attended volunteer scfaoola scored significantly higher on adilevenent 

*° attend school; for older students (aged 11-17) 

the differences were substsntlsl. There exists more evidence on the 
effects of attendance or nonattendance In the literature on the 
effectiveness of ITV; Cfau and Schraan 11967] reviewed nine examples of 

coBpared ITV with no Instruction and in all nine those with 
ITV performed better. This stands In contrast to the typical "no 
significant difference" that predoadnates covparlsons of ITV with face- 
j-face Instruction. For a further discussion of the affects of sdiool 
«^tendanca see Jencks [1972, pp. 85-89]. 
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III. INSTRUCTIONAL RADIO 



Beginning In the 1920 's. inatructional radio was widely used 
in the United States, but with the advent of television and adverse 
regulatory decisions its use here dwindled as it did. to a lesser 
extent, in other developed countries. Developing countries.^ however, 
make increasing use of ra-lio and. as our evidence suggests that radio 
can be effective instructionally. there my be an Important role for 
it in the developed countries as well. Its principal attraction lies, 
of course, in its cost, which is low when compared to television. 

Early attempts to use radio for instructional purposes were 
rarely subject to systematic evaluation and. since IR has been used 
infrequently in the United States recently, available evaluation 
material is limited. For this reason the present review begins by 
providing evidence on the extent to which IR has been used in variots 
countriee a« indirect evidence that it has some value. Then the 
conclusions of two earlier surveys on the effectiveness of IR are 
reported and. finally, several examples evaluating IR and audio record- 
ings are presented in more detail. 



Use of IR 

Atkinson [1942 (a). 1942 (b)] provided journalistic information 
; a a substantial number of IR projects undertaken in the United States 
prior to 1939; his books provide information concerning the operational 



u e»«Ple. the Nor York Tinea of August 22. 1972 reported that 
Snanghal is tuning In the radio dally and gleefully learning to say 
Hello . For the preceding five months, a half-hour English lesson had 

been broadcast three tlmea dally and had met with great popular succeas. 

sadio baa a hlatory of use for education in China; Chang [1936] reported 

oiat its use waa one of three coio^onents of a mass education pxoeram 

then underwqr In China. 
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problems and history of early uses of the medium in this country. 
Skomia [1962], Saettler [1968], and Wrlghtstone [1952] described the 
later evolution of Instructional radio In the United States, and 
Wrlghtstone provided a valuable suniaary of early research concerning 
Its impact. Though It is not extensively used at present, a number 
of school districts do continue to use radio. 

In Britain radio has been used extensively to provide school 
broadcasts. Currently 63 educational radio series are broadcast to 
schools in England. Almost all of these series use Illustrated pupil 
pamphlets to support the lessons at the reception end. Within Britain, 
school broadcasting emphasizes collaboration between the classroom 
teacher and the radio teachers. Kadlo primarily provides lessons which 
the children might otherwise not receive, such as art, music, and 
foreign languages. 

Australia also makes use of instructional radio broadcasts in its 
schools — see Bull [1960] or Klnane [1967] , In 1960 over 90 percent of 
the schools received some radio lessons. Curriculum enrichment broad- 
casts, similar to those of the BBC, are used in the urban schools and 
even more extensively in the one-room rural schools. At the higher 
education level, the Radio University, of New South Wales, enrolled 
over 6000 students in 1965. One particularly inventive instructional 
radio program was originated by Miss Adelaide Miethke, a well-known 
educator from the state of South Australia. She arranged to use the 
shortwave services of the Royal Australian Flying Doctor Service to 
communicate with students in the isolated "outback" regions, and each 
outback camnunity purdiased a transmitter. So, for a limited period 
each day, the stiadents are able to talk to a teacher and to each other 



The more active stations using IR in the United States at the 
present time Include KRVM (Eugene, Oregon), WGBO (Newark, New Jersey), 
KSLH (St. Louis, Missouri), KANW-IM (Albuquerque, New Mexico), KBFS 
(Portland, Oregon), and WYNE-FM (New York, New York). Kottneyer [1970] 
reported that rhe KSLE program In vocabulary Improvement, a si^plonent 
to traditional Instruction, resulted In substantial gains In IQ and 
spelling over controls from previous years. Evaluation material on the 
other programs wos unavailable to the present authors. 
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about their correspondence lessons. The interest in IR in Australia 
dates back to at least the I930's. One of the first statistically sound 
IR evaluations was undertaken there, by Thomas [1937], and he reported 
no statistically significant differences in achievement in most cases. 
There was a tendency for the TI students to do slightly better on an 
immediate post-test and for the IR students to do better after a delay. 
The amount of exposure to IR was, however, small. 

Another country making widespread use of instructional radio 
Is Japan. In 1935, Nippon Hoso Kyokai or the Japan Broadcasting 
Corporation (NHK) began a small program of radio broadcasts to the 
school [Hatono, 1960; NHK, 1964]. After World War II, a decision was 
made to modernize completely the Japanese educational system, in terns 
of both curriculum and teaching technique. Radio played a large role 
in this nodemization in compensating for the many textbooks lost 
during the war and in rapidly disseminating the new methods of instruc- 
tion. A 1958 survey by the Broadcaatlng Culture Research Institute of 
the NHK reported that 47 percent of the primary schools, 37 percent of 
the lower secondary schools, and 27 percent of the upper secondary 
schools regularly used radio broadcasts. In Japan it is possible to 
receive a secondary level diploma without attending a classroom through 
a combined program of correspondence courses and radio lessons. 

One of the more successful uses of radio In a developing country 
has been in Thailand when broadcasts to the schools began in 1957 and 
by 1965 reached over 800,000 students with lessons in English, social 
studies, and music. Students receive an average of 10 to 30 minutes 
of Instruction weekly in each subject, as supplements to their regular 
lessons. Schramm [1967] suanarlzed the Thai experience and reported 
on a 1959 etvaluation by the Thai Ministry of Education. The evaluation 
showed students who received the radio music supploaents to be sig- 
nificantly superior (p - .001) along several dimensions to student wsd 
did not; the English lessons shoved no such consistently positive 
effect and were subsequently extensively revised. Perhaps most 
interesting were the lessons in social studies, the purposes of which 
were to Inculcate socially desired values. The Ministry evaluation 
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concluded that this objective was being met since a significantly 
higher percentage of radio students expressed agreement with desired 
attitudeb and values in a questionnaire. 

These descriptions provide only a sample of the Instances in 
which radio has been utilized abroad; Williams [1950], Bereday and 
Lauwerys [I960], and Leslie [1971]^! described additional examples. 



Surveys of IR Evaluations 



Two surveys review inforMtlon relevant to the effectiveness of 
IR. One is Section VI of Chu and Schramm's [1967] comprehensive review 
of learning by television. The second is a position paper by Porsythe 
[1970] that, in an earUer form, was prepared for the President's 
Connission on Instructional Technology. Sources of further information 
on IR may be found ia a 432-entry indexed bibliography compiled by 
R. Madden [1968], and an early review of research undertaken primarily 
in the late 1930 's and early 1940 's may be found in tfoelfel and Tyler 
[1945]. 

Chu and Schramm [1967] numbered the principal conclusions of 
their extensive survey. The ones aost relevant to IR follow. 

"53. Given favorable conditions, pupils can leara fnm acy 

instructional media that are now available. 
"58. The use of visual laages will Inprove learning of manual 
tasks as well as other learning where visual Imagea can 
facUlUte the association process. Otherwise, visual 
linages may causa dlatractlon and Interfere with learning. 
"60. Student response la effectively controlled by programmed 
■ethods, regardless of the Instructional medlm." 
Their general conclusion Is that radio, particularly when appropriately 
supplemented by visual material, can taach effectively and, for 
uny purposes, as well as otbar aedla. 



erJc 



paper by'SjlS!"'" '° unpolished 
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Forsythe [1970] reached a similar conclusion. In summarizing 
studies of radio's effectiveness he concluded: 

"Research clearly Indicates that radio is effective In 
instruction. Experimental studies comparing radio teaching 
with other means or media have found radio as effective as 
the so-celled 'conventional methods.' Even though radio has 
been criticized for being only an audio medium, studies have 
shown that visual elements in learning are not uniformly Im- 
portant. In many educational situations visuals may be more 
harmful than helpful. Also, the efficiency of combined audio 
and visual media has been challenged by stud'.M which show 
that multi-channel comunications may not be inherently more 
effective than single channel presentations." 
To support his conclusions, Porsythe listed, among others, studies of 
Carpenter [193A], Cook and Nemzek [1939], Harrison [1932], Heron and 
Ziebarth [1946], Lumley [1933], Miles [1940], and Wiles [1940]. He 
also mentioned two experiments by NHK In Japan [NHK, 1955, 1956] that 
favored radio. Forsythe, along with Chu and Schramm, concluded that 
IR compares well with TI. It should be kept in mind, though, that 
most of these studies are old, and that in many of them the statisticel 
controls were imperfect, the amount of Instruction carried by IR vas 
small, or the classroom teacher did participate in the program. 
Nonetheless, we beUeve that the overall conclusions of Chu and Schramm 
and of Forsythe are consistent with the available evidence. We also 
feel that there is substantial value, particularly for developing 
countries, in obtaining much more extensive evidence on the effectiveness 
of IR; of particular Importance would be experiments using IR to carry 
the bulk of Instruction In one or more subject matters for periods of 
at least one academic year. 



Specific Evaluations of IR 

To give a more concrete Impression of the results of this research, 
this subsection discusses several of the better studies in more detail; 
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these Include studies that compare IR with ITV as well as some tha-. 
compare Instruction by audio tape to TI. 

McLuhan [1964] summarized an Interesting study In which four 
randomized groups of university students were given the same Infor- 
mation about the structure of prellterate languages. One group 
received It via radio, one by TV, one by lecture, and one read It. 
In all cases the Information was given In a straightforward manner, 
unembeUlshed m h teaching aids. The first results Indicated that 
the students learned more from TV and radio teaching than they did 
from lectures and print, and that the TV group stood above the radio 
group. However, when the experiment was repeated using improved 
auditory and visual alda, the relative effectiveness of the different 
media changed. Television and radio once again ranked above lecture 
and print. Unexpectedly, however, radio stood significantly above TV. 
In this experiment, TV seemed to fare less well as a teaching medium 
because of limited audience participation; better results were obtained 
with IR because of efforts to engage the students (asking them to look 
at certain illustrations, etc.).^^ 

One interesting and detailed evaluation is an early study of 
the Wisconsin Research Project In School Broadcasting [1942] of radio 
lessons in music. A music course was first broadcast in 1922 and an 
evaluation was undertaken In 1929. This evaluation indicated that the 
Bwslc course was highly successful and so, in 1931, the Wisconsin School 
of the Air began a series of weekly broadcasts called "Journeys in 
Music Land," the effectiveness of which was studied during 1937 and 
1938. The students who participated in these radio classes were in the 
fifth and sixth grades in both rural and uiban schools. By March 1, 1938, 
there were 814 listening classes in 770 schools. The aim of the broad- 
casts was not only to teach music appreciation, but also to teach 
children to sing and read music. The broadcasts were planned around 



Sfithout formal evaluation Skornla [1968] reported that in Holland 
and the Scandinavian nations IR had been found better than ITV for some 
aubjects when cKerdaa aanuals and other student participation materials 
were used simultaneously tritik the radio lesson. 
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a minimum of assistance by the classroom teachers, because aost of the 
teachers had no skill In musical Instruction. The first year of the 
experimental music culminated In a radio music festival In May 1938 
and the evaluation states: 

"It was evident to all who heard the Radio Music Festival 
that the children had learned to sing with clarity of diction 
and beauty of tone. Their enthusiasm and their enjoyment of 
song made the whole performance Impressive." 
The more systematic evaluation of effectiveness compared 12 
classes that listened to the music broadcasts with 8 comparable classes 
whose teachers pursued the stated alms of the radio broadcasts » but 
did not use the broadcasts tihemselves. The classes were matdied on 
the basis of number of students, teacher competence, and available 
naisical equipment. Measures of the students* sex, giade level, previous 
musical training, chronological age, and mental age were also made, 
although these were not used Initially to match the experimental and 
control classes. When examined, however, these factors did not affect 
the amount of student gain. The exper'jaental period lasted 15 weeks 
during which classes In both groups received a total of 75 minutes of 
Qusical instruction each week. For the IR classes this was divided 
between a 25-minute broadcast once a week and 40 minutes of supplanentary 
classroom practice. 

Several tests were devLsed to measure the gains of the students. 
The measure of singing quality shoved no differences between the 
radio and control classes except that the IR classes maintained better 
rhythm. On ability to sing an unfamiliar song at sight, the initial 
scores of t^e IR classes were significantly lower and their gains were 
significantly greater. Again, the most significant difference was in 
ability to maintain correct rhythm. The IR classes and control classes ^ 
did not differ significantly in thair initial ability on the test of 
technical skills. On the final test, however, the IR classes scored 
significantly better in their ability to recognize note values, read 
at sight, and recognize xfaytbms; there was no significant difference 
between the IR and control classes in ability to take musical dictation 
or staff dictation. 
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The Wisconsin Research Project evaluated six other radio series 
in addition to the music one ve have Just described, and the volume 
they produced remains perhaps the best single source of evaluative 
material on IR. While the other studies they reported were less 
favorable to IR than was the music evaluation, they provided ample 
evidence for thr capability of IR to carry important segments of the 
curriculum. 

Several more recent studies that were carefuJ.ly controlled examined 
the effect of substituting an audio-tape presentation for live lectures. 
Popham [1961] divided an introductory graduate level course into two 
sections. In one he taught in a lecture-discussion fomat; in the 
other, he played a tape-recorded version of the lecture and then led 
a brief discussion period. The two sections were matched on scholastic 
aptitude and two achievement pretests; on several posttests. Popham 
found no significant differences between the two sections. In order 
to test the importance of having the course instructor present for the 
discussion sessions. Popham [1962] performed a similar experiment in 
which lectures presented by tapes were followed by a discussion led by 
a relatively untrained student. Again, no significant differences 
between conventionally taught and tape-taught students were found. 
In t Tperlments students had generally favorable attitudes toward 
instruction by audiotape. They felt the lectures were better organized, 
and they felt freer from distractions. However, they were dissatisfied 
with their InablUty to question or disagree wltU the Instructor during 
the lecture. 

Menne. Kllngensmltfa, and Nord [1969] extended Pophmn's work 
by providing each student with a tape recorder and a complete set of 
taped lectures that allowed each to work at his own pace. They recorded 
and edited lectures for an introductory psychology course taught 
every quarter at Iowa State University. The blackboard notes from 
the lectures were prepsred In booklet form. For two academic quarters 
they compared students who took the course solely from audiotape with 
students who took it from the lecturer from whose earlier lectures the 
sudlotapes had been prepazed. A total of 290 students elected to take 



- 25 - 



the course by tape, while 408 chose the live lectures. In spite of 
the self selection, the tiro groups were closely matched in terms of 
their high school rank in class and measures of achievement and 
scholastic cptitude. 

In terms of posttest scores and final grades, the two groups 
did not differ significantly. When comparisons were made in terms of 
groups ordered (into quartiles) by high school rank in class, there 
was a clear advantage to using tapes for the lowest quartile; for the 
others there was no difference. A possible explanation might be that 
the poorer students were able to listen more than once to lectures they 
had failed to understand the first time. A final interesting difference 
between the two groups was that only five of the students learning by 
tape dropped out, whereas 58 attending the lecture sections dropped 
out. Menne et al. [1969] speculated that it is less likely that 
students will fall irremediably behind if the tapes are always at hand. 



Conclusions 

Radio has been used extensively for formal classroom Instruction 
in the United States (more In the past than at present) and elsewhere. 
There exist, however, only a limited nuiii>er of good evaluations of 
the effectiveness of IR. These evaluations indicate that IR (supple- 
mented wltii appropriate printed material) can be used to teaCh most 
subjects as effectively as a live classroom Instructor or ITV. Due 
to the limited number and scope of good evaluations now available, 
and to the potential economic significance of IR for developing 
countries, much more research ~ both survey and experimental -- is 
highly desirable. 



IV. INSTKDCTKWAL TELEVISKW 



This section Is briefer than the others, because two thorough 
and recent reviews of the literature on the effectiveness of ITV 
already exist: Chu and Scfarana's [1967] Learning from Televisio n: 
What _die Research Says , and Dubln and Kedley's [1969] The Medium May 
be Related to ^ Message: College Instruction bjr TV. Conclusions 
of these reviews are suBnarlsed first with respect to achievement and 
then with respect to attitudes toward the use of the mediiBD.^^ The 
present review does not cc-ver the literature on the instructional 
use of film because of its close similarity to ITV; for a good over- 
view of the research on film see Allen [1960. pp. 116-118]. 

ITV and Student Achievement 

Chu and Schraaa surveyed 421 comparisons of ITV with TI that are 
reported in 207 separate studies. Tables 3 and 4. reproduced from Chu 
and Schramm, suonarixe a mnber of their findings on the relative 
instructional effectiveness of die two media. Table 3 Indicates that 



IX. 

jVo recent projects not covered in these two previous surveys 
are worth mentioning. During tiie last few years probably the woat 
intensive evaluation of an ITV project was initiated and is now almost 
complete. This was a U.S. Agenqr for International Development funded 
evaluation of the educational reform «nd introduction of ITV into 
grades 7-9 In El Salvador. Schramn [1971] provided a sunary oj' that 
researdi to date; more detailed information may be fotmd in McAtany, 
Mayo, and Homlk [1970] . In a second project, at the postgraduate 
level, Colorado State University provides M.S. level courses to 
engineers at corporations and government researdi laboratories tlirough- 
out the State of Colorado. Over 12,000 quarter hours of university 
credit were earned and 24 M.S. degrees awarded through this program 
to date. For a discussion of evaluation and costs see Bald:dLn, Davis, 
and Maxwell [1972]. 
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Insert Tables 3 and 4 about here 



students at all grade levels learn well from ITV, though this seems 
somewhat less true for older students than for younger ones. Table 4 
Indicates that the effectiveness of ITV cuts across virtually every 
subject natter. 

Dubln and Hedley [1969] provided a more detailed survey of the 
effectiveness of ITV at the college level. They reported on 191 com- 
patisons of which 102 favored ITV and 89 favored TI, although most of 
the differences were Insignificant at standard levels of statistical 
significance. When data were available, Dudley and Hedley extended 
their comparisons to Include the distribution of the t^ statistics of 
the Individual comparisons of ITV and TI; In this way It was possible 
to weight appropriately differences in performance of differing degrees 
of statistical significance. The results of this analysis, applied to 
all their data, indicated a sli^t, but statistically significant 
difference in favor of TI. When studies of two-way^* TV were dropped 
from the sao^le, the overall con^arison yielded a small, statistically 
insignificant advantage for TI. Figure 1 shows the distribution of t^ 
statistics for this sample. 



Insert Figure 1 about here 



An unusually stringent criterion for interpretability of results 
was utiUsed by Stickall [1963] In comparing ITV to TI, and it is worth 
cammenting on his sunrqr here. After wrT.lnlng 250 comparison of ITV 



Two-way TV Incorporatas an audio-return capability that allowu 
students to ask quasttoos during a live ITV broadcast. The 26 com- 
parisons of this mode of iaatruetlon with TI yielded a highly 
significant advantage for TI. 



TKBLE 3 

Results of k21 Comparisons Between ITV and TI 
[Chu & Schramn, ISSj] 



Level 


Nxanber of cases of 


No significant 
difference 


IT7 more 
effective 


TI more 
effective 


ELementary 


50 


10 


k 


Secondary- 


82 


2k 


16 


College 


152 


22 


28 


Adult 


_2k 


J. 


__2 




508 


63 


50 



TABLE k 

Relative Effectiveness of lOV and TI, by Subject hfatter 
[Chu & Schramm, 1967] 



Subject 


Nuniber of 
ccmparlsons 


Percentage of cc»nparlsons 
in irtiich ITV did as veil 
or better than TI 


Mathematics 


56 


89.2 


Science 


100 


86.0 


Social studies 


77 


89.6 


Himanltles 




95.5 


Languages 


77 


88.3 


Skills 


26 


96.1 


Miscellaneous 


ko 


75.0 
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FAVORS 
FACE-TO-FACE INSTRUCTION 



FAVORS 
ETV 



20- 




10- 



STANDARIZED DIFFERENCES 
N»67 
MEAN »- 0.03 
SD« 1.59 
t»O.I6 P > 050 



Fig. 1. One-vay nv compered to TI, Independent caaperieona, 
(Adapted trm Diibin & Bedley [1969I, Figure 3, p. 19. The measure on 
the horisontal axis is the value of the t ratio for the comparison.) 
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to TI Stlckell found 10 studies that fully net his requirements for 
adequate controls and statistical nethc 1 (InterpretabiUty) and 23 
that partially met his requirements. Schramm [to appear] provides 
clear tabular summaries of these studies None .f the fully Uit^r- 
preL«ble studies 1 of the partially lnterpr,^i«h Ip o,,p<, flhe«,.rl 
statistically significant differences; each of the three Htatlstlca I ly 
Significant cases favored the ITV group. It should perhaps be notc-d 
that when highly stringent controls are Imposed on a study, the nai ur,- 
of the controls tend, to force the methods of presentation into hu<I. 
similar formats that one can only expect the "no significant dl rferc.„< <.„" 
that are in fact found, when ITV is used in a way that takes advantages 
of the potential the mediun offers - as. perhaps, with Sesame Street^^ 
- we would expect more cases of significant differences between the- 
experimental group and the "alternative treatment" (for it would not 
be a "control" in Stickell's sense) group. 



Attitudes Toward ITV 

Chu and Schramm sumnarized their conclusions in a series of 
numbered paragraphs. The ones relevant to attitudes are quoted below. 
They noted at the outset that "the research evidence makes attitudes 
toward instructional television seem rather more favorable than one 
would expect from the experience reports that circulate. Regardless 
of this evidence there Is good reason to think that some resistance 



15. 

viawlna n»t y'T"? pre-school age children, and for 

viewing out of school. Sesame Strset falls outside the scope of this 

rZZ\ f ^Tr?:,?? °^ Ses«ne S tree? - see 

Sflct ^ V^K" it had a significantly positive 

effect on disadvantaged pre-school age children in terms of a large 

ifv f ^P^ci^i*^ Soais the producers set for the program. A 
p^oblem with the first year's evaluation was that there may have been 
a correlation between frequency of viewing end other variables tending 

year by facilitating and encouraging viewing by a randomly chosen half 
of the subjects and not doing so for the other half . 



among teachers has been aroused wherever and whenever television has 
been Introduced for purposes of direct teaching." Their nianbered 
conclusions are as follows; 

"37. Teachers and pupils are more favorable toward the 

use of ITV in elementary school than In secondary 

school and collie. 
"38. Administrators are more likely to be favorable toward 

ITV than are teachers. 
'*40. At the college levels students tend to prefer small 

discussion classes to television classes, television 

classes to large lecture classes. 
"41. Favorable attitudes are distributed widely enough 

among different televised courses to cast doubt 

on the assumption that some academic subjects, per 

se» may be disliked as material for ITV. 
"42. There is evidence of a Hawthorne effect among students 

beginning to use ITV» but no firm evidence that 

attitudes toward the medium necessarily improve 

or worsen with time. 
"43. Liking ITV is not always correlated with learning 

from it/' 

Dubln and Hedley presented a slightly more optimistic view of 
attitudes toward ITV by college professors and students • Professors > 
they found » are generally favorable toward ITV though a substantial 
majority of them would rather send their own children to a university 
using TI rather than one that vas otiiexwise similar but that 
used ITV for its large introductory classes. Junior faculty and 
facility who have taught a xxuater of large lecture classes tend to 
favor the Introduction of ITV. 

Dubln and Hedley also reviewed a nvaber of studies on the 
attitudes of college students toward ITV. Students have more favorable 
attitudes toward ITV after they have experienced it than before; after 
exposure to ITV half to tvo-thlrds of the students surveyed reported 
attitudes that w^.re favorable (as opposed to neutral or unfavorable). 
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Aaked whether they would choose ITV or TI, less than one-third indicated 
a preference for ITV (and here there la substantial variation among 
Institutions). If, hatie^.r, the choice was betueen ITV and TI In the 
fonn^of a large lecture course, typically over half the students preferred 
ITV. Dubin and Hedley concluded that "the college student as consumer 
of teaching does not exhibit any significant resistance to the intro- 
duction of educational television into his ovn Instructional program. 
He will take whatever method or medium of Instruction la offered, damn 
or praise it on Its merits, and get on with the business of pursuing 
his college education [p. 86]." 

In a particularly Interesting study Greenhill, Carpenter, and 
Ray [1956] examined perhaps the best indicator of students' attitudes, 
their own free choices. In a university level chemistry class 312 
students were required to attend lectures for five weeks In the large 
lecture hall and for five weeks in a relatively small TV classroom. 
The students were then given their choice concerning which way to 
continue the course; about one third selected TV. A large fraction 
of students had no strong preference. 



Conclusions 

rrv can teach all grade levels and subject matters about as 
effectively as TI, though sobs evidence Indicates that it performs 
relatively better at lower grade levels. A significant fraction of 
teachers and students have Initially negative attitudes toward ITV; 
these negative attitudes tend to lessen, but not necessarily disappear, 
with time and appropriate administrative behavior. Evaluations that 
report "no significant difference" between ITV and TI are usually based 



^inane [1967] reported that students in a calculus course at 
the Australian Radio University (which also utilises ITV) expressed a 
strong preference" for the television over the radio version of the 
course. 



lERJCi 
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on experlaental designs that hold almost everything but the medluB 
constant. It Is plausible - though not. to our knoirledge. exper- 
laentally verified - that attempts to use the distinctive potential 
of the television aediuB would result In nore systematic findings of 
significant differences between ITV and alternative treatment groups. 
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V. PROGRAMMED INSTRUCTION 



Although In recent years the Intensive evaluation of PI has 
considerably leasened, over the past 15 years many evaluative studies 
have been isade. We review a nunber of thcan briefly In this section. 
We first state the conclusions of several previous reviews of the 
literature then summarize a nunber of more recent studies . There 
exist several valuable anthologies of papers on PI - including Lumsdalne 
and Glaser [1960], DeCecco [196A (b)], and Glaser [196:i - and the 
Interested reader is referred to these for useful source -naterials. 



Previous Reviews 

After a review oi T, field ecperiments , Silbennan [1962] found 
that all of them showed t^at PI took less time to complete than TI. 
Rirtheimore, in 9 of the staii « students in the PI groups scored higher 
than their counterparts. 1« the other 6 studies there was no diiference 
between tiie two approaches. 

Another good survey ot the earlier research on programed 
instruction Is Schranin [1964]. Schranun introduced an annotated bibliog- 
raphy of approximately 190 rest arch studies In the area of PI with a 
summary evaluation of those 8tu,lles. Thirty-six of the studies he 
reviewed compared PI with TI; of these, 18 showed no significant 
difference in performance between the PI anr! TI groups, 17 showed a 
significant si^eriority for PI, and only one showed superiority for TI. 
In 8 of die studies PI students needed less time to complete the require.! 
materials than did the TI students. Schrann also reviewed the evidence 
concerning the Importance of such variables as sequence ordering, step 
length, error rate, con8t;£^ucted responses, and feed>ack to the student 
on tlie accuracy of his answers. 

In a more recent review, Lange [1972] r«»ported that between 
960 and 1964, 112 conparative studies were conducted that aimed at 



matchxng PI and TI. Of these studi.es 41 percent showed PI to be 
superior. 49 percent found no difference, and 10 percent found PI to 
be worse than TI. The studies he revlei^ed are not. however, completely 
Independent of those reviewed hy Schramm. 

Zoll [1969] provided a fairly extensive review of research in PI 
in mathematics. He undertook a review of 35 studies reported in the 
literature, many of them In th« form of dissertation abstracts. He 
cited 7 studies that specif Icslly evaluated one or more of the 
commercially produced PI programs in comparison with TI. Of these and 
the other studies the most omwon conclusion is that no significant 
differences were apparent. It is important to realize that such results 
are standard m the mathematics education literature. One explanation 
Is that the variance in InJividual ability and achievement is large 
enough to make it dlfficuJ.t to establish significant differences due 
to different methods of liistruction. Another possible explanation, 
consistent with Silberman's findings, la that while student achieve- 
ment may not significantly differ, less student time may be required 
with PI. (Lumsdalne [1963. pp. 611-613] discusses the Importance of 
time as an instructional variable.) Of the 35 studies reported by Zoll. 
10 included results from attitude questionnaires on student reaction to 
PI. While responses were generally favorable, three studies [Alton. 
1966; Little. 1967; Meadowcraft. 1966] indicated that Interest decreased 
with time. 

Along related lines. Peterson [1972, unpublished] surveyed work 
in the area of mastery learning. Hastery learning is a general term 
used to describe a programmed Instructional process in which a subject 
matter is subdivided into many smaller unite and each student attains 
a mastery of a specific unit before beinfj advanced to the next unit. 
Advancement is based on the percentage of correct responses on a test of 
the current unit. A variety of materia. s may be used in the teaching 
of the subject matter Including audio-visual methods, tutorial help, 
workbooks, games, and small group study. Peterson surveyed a total 
of 21 studies in mastery learning; sane of the studies repotted results 
of more than one experiment. AchLsvement measures included grade in 



courst (X A or E) tnd scores on post-tests. Coaparisons xrere made with 
traditionally taught courses and twenty-four of the e3cperliT»ents 
favored tnrstery leaning while there was no difference reported in 
four of the experiments. 



Specific Studies 

It is natural then to ask in what areas is PI effective and for 
whom? A wide range of examples on the use of PI may be cited. Brigham 
[1970] used prtgranmed texts to teach woodwind fingering; Bullmer .ri972] 
used progTiimmed materials to teach accuracy of interi-ersonal perception; 
Ashford [1968] uied PI to teach fundamental concepts of music theory and 
found that three years after the 11-week course, on a recall examination, 
students in the PI group performed better than students receiving TI. 

In an extensive study. Johnson [1966] compared three different 
programmed textbooks and two conventional texts in 21 elementary 
algebra classrooM. The texts and programs were all prepared as part 
of the School Mathematics Study Group project. He found that one of 
the texts was the most satisfactory for each of the three abiUty levels, 
high, middle and low. but good achievement results were obtained by both 
high- and mldd].e-abillty-level students using the PI units. 

In a study concerned only with low arlthrr.* .c achievers. Tanner 
[1966] found no differences In achievement between seventh-grade students 
using PI under teacher supervision and students receiving TI. In the 
same spirit Bobler [1965] found no significant differences among twelfth- 
grade students using either PI or TI to improve weaknesses In arithmetic 

ski:- 

Another area of research concerns the effects of individual 
differences. The finding seens to be In general that the Intelligent 
students [WilUams. 1963. 1965] and the creative students [Tobias. 1969] 
profit oore than other students In texms of speed of learning and posttest 
scores. However, these results are hardly surprising for we would expect 
such results from almost maj form of Instruction. 
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Results of somewhat greater interest are the findings by Shrabel 
and Sassenrath [1970] that anxious students outperform students with 
low anxiety, that an easy program with short steps Is better suited to 
persons who are low on need for achievement and high on fear of failure 
or text anxiety, and that a hard program with long steps Is preferable 
for those with a high need for achievement and low fear of failure. 
However. Tobias and Abramson [1971] failed to replicate this anxiety 
finding. 

In one of the largest studies to date. Attlyeh. Bach, and Lumsden 
[1969] reported on an experiment In Introductory economics that was con- 
ducted simultaneously in 48 colleges and universities and that involved 
over 4000 students. The students were divided into three groups. The 
first group studied a programmed text during, on the average, the first 3 
wjeks of the term and attended no lectures. The second group supplemented 
regular Instruction with a programmed text, and the third group served 
as a control. The results of the experiment were analyzed by multiple 
regression to control for differing characteristics of the students and 
the schools they attended. Of the two programmed texts used, one proved 
to be significantly better than the other. For the superior text, students 
in the first group who only read the programmed text did less well (but 
statistically insignificantly so) than the controls in the third group; 
students la the second group who supplemented their regular course with 
that programmed text did significantly better than controls. The first 
group, who read the programmed text only, experienced a substantial time 
saving. It is of interest that this study, by using two separately 
prepared sets of progranmied materials, illustrates the difficulty of 
simply comparing media without simultaneous consideration of content. 
Use of the poorer programmed text as a supplaaent actually weakened the 
performance of students. 

Another example of one of the better studies on the effectiveness 
of PI is Doty and Doty [1964]. These authors studied the effectiveness 
of a prograaned unit on physiological psychology for 100 introductory psy- 
chology students [KJmble, 1963]. The program had 1.507 frames, was 
assigned as required outside dasszoon work and was not discussed In 
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any class meetings. Students vere given two weeks to complete the 
program, after which they were tested by means of a 75-itaa multiple- 
choice achievement test. The scores on the test were used as the index 
of PI effectiveness. 

Doty and Doty were Interested in the Intercorrelations between 
the PI achievement as measured by the test and student characteristics. 
The following student characteristics were studied: Academic ability as 
measured by cumulative CPA; achievement motivation as measured by the 
Edward's Personal Preference Schedule. Achievement Need Scale Scores; 
creativity as measured by means of Getzels' and Jackson's four tests of 
creativity; social need as measured from scores on the Gullford-Zlj«nenaan 
Temperament Survey. Achievement on the PI unit was found to be signif- 
icantly related to GPA, social need, and creativity; the correlations 
with creativity and social need were negative. 



Conclusion 



In evaluating the effectiveness of PI for use in various educa- 
tional settings, the study of Doty and Doty suggests the kind of research 
required la the future. A better understanding Is needed of how student 
personality variables differentially relate to achievement in PI . More 
generally, the current research emphasis in PI seems to have dianged 
from direct comparative studies of effectiveness to detailed studies of 
how to Improve the programs, how to Increase student interest, and how 
to adapt PI to unusual educational settings. In the meantime, on the 
basis of the research to date, it Is reasonable to conclude that PI Is 
generally as effective as TI and mqr result In decreasing the amount of 
time required for a student to acbleve specific educational goals. 
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VI. COMPUTER-ASSISTED INSTRUCTION 



Among the alternative instructional media considered in this 
survey, not only is CAI the newest, but in terms of the initial cost 
of instruction per hour, it is also the mast expensive. At the same 
time, however, this technology provides the richest and most highly 
individualised interaction between student and curriculum of any of 
the methods of instruction yet developed. 

Prior to the early 1960 's projects in CAI were virtually unknown. 
Until 1970 or 1971, aLnosc all the projects were developed in university 
researdi settings, especially in universities with rich computer 
resources. In the last few years, however, a number of school dlstrf.-ts 
have begun to run their own CAI courses, and even thougli it is not 
presently possible, extensive evaluations of CAI, separated from the 
stlmulitt and supervision of a computer-based research center, should be 
available in the near future. 

Given the data collecting and analyzing power of computers , it 
Is surprising that more recorded evaluative studies on the effective- 
ness of CAI are not available In the literature. Part of the expla- 
nation is probably that during the first years of developing this nev 
method of instruction the main efforts have gone toward solving the 
technical problems associated with the ongoing operations and only in 
the last several years have there been adequate time and opportunity 
to make systematic evaluative studies. Evaluations of the effectiveness 
of CAI programs have, nonetheless, been conducted for most levels of 
education. The most Intensively researched area is that of the 
effectiveness of drill and practice programs In elementary laathematics 
and reading, and we begin our survey with a review of that research « 
He then turn to a ninber of studies conducted at the college level. 
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Elementary-s chool Drill and Practice 

The available CAI drlll-and-practlce programs provide a supple- 
ment to the elementary student's regular Instruction In mathematics or 
language. Several times a week the student receives sessions at the 
CAI terminal that last about 10 minutes; these sessions provide intensive 
drill in the concepts he is learning in his regular class. Vinaonhaler 
and Bass [1972] recently surveyed over 30 separate experiments (involing 
a total of about 10,000 studcmts) that compared TI to TI augmented by CAI 
drill and practice at the elementary level. They concluded that "... there 
appears to be rather strong evidence for the effectiveness of CAI over 
traditional instruction where effectiveness Is measured by stapdardized 
achievement tests." In this survey «e review several of those experiments 
and then summarize several other studies that attempt to relate amount 
of achievement gain to amount of CAI. 

Suppes and Mornlngstar [1969] reported the results of the evaluation 
of drlll-and-practlce programs for schools In California for the 1966-67 
and 1967-68 academic years and for schools In McCoai>, Mlsalssippi for 
1967-68. The programs they discussed and analyzed for supplementary drill 
and practice were given to the students on a dally basis. Students spent 
not more than 10 minutes a day at teletype terminals connected by phone 
line to the computer at Stanford. 

To evaluate the effectiveness of the drlll-and-practlce programs, 
they adainis tared the arltimetlc portion of the Stanford Achievement 
Test to both experimental and control clasees, using different fonw 
for the pretest and the poattest. Tests were given in four California 
schools for the 1966-67 evaluation. The precest, posttest, and differ- 
ences for experimental and control groups on the computation sect:'ons 
are whom in Table 5. Table 6 shws slmilfir data for the students from 
Misalaaippi for the 1967-68 school year. 



Insert Tables 5 and 6 about here 



We shall not discuss all the results here, but several points 
of interpretation are significant. At the end of the school year the 
investigators learned that at one control school in California teachers 
and administrators had added 25 minutes per day of classroom instruction 
and practice in arithmetic for grades 4 and 5. Dat-. from this control 
school are responsible for the negative t value at grade 4 in Table 5. 
What is Important is the demonstration that with a sufficiently intensive 
effort the effects of classroom drill by the teacher can be as effective 
as drill and practice on a computer. Drill and practice on the computer, 
however, took less time and did not require an additional effort from 
the teacher. A secojid point of Interest is that the CAI results for 
Miasissippi (Table 6) are substantially more Ijnpresslve than those for 
California. This is an example of the generally noticed result that CAI 
drill and practice Is more effective with students who start below grade 
level. 

A different approach tried in the New York City Schools is the 
Dial-A-DrlU program In which students are called at hone and given 5 
minutes' practice In oral arithmetic problems. The oral exercises are 
generated from digitized word recordings stored on a computer disk, and 
the students respond by using a touch-tone dial. Students in grades 2-6 
participated in the demonstration project. Except at the third-grade 
level, students received the program at most three days a week. An 
intensive program for third graders required their receiviag 5 minutes 
of drill and practice six days a week. Because the project was supported 
by tm Urban Education Grant, tiie students participating were mainly from 
disadvantaged environments. 

Evaluation of the Dlal-A-Drlll is reported la Beech, McClelland, 
Horowitz, and Porlaao [1970]. The results may be sumnarlzed briefly 
as follows. Experimental and control groups were both given the 
Metropolitan Achievement Test of Arltimetlc Computation and a specially 



For further discussion of neasurement msthods and eapirleal 
results concerning Inequality-reducing aspects of CAI see Jamison, 
Fletcher, Suppes, and Atkinson [1971] or Fletchsr and Jamison [1973], 
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designed Oral Arlthaetlc Teat in October 1969 and May 1970. A least- 
squares analysis of covarlance of the 1970 arithmetic achievement data 
failed to produce statiatlca].ly significant differences between the 
experimental and control students at any grade. Further analysis of 
the data showed that some students in the program did not actively 
participate. 

A separate analysis was performed on students in the experimental 
group who had more than 32 sessions (approximately one per week) , and 
those selected students were Tiatched with control group students . Three 
tests for correlated means were performed and only third-grade students 
exhibited a statistically significant difference. This difference was 
on the arithmetic test, in which the experimental students performed 
better than the control students. One inference to be made from this 
study is that 15 minutes a week, that is, three sessions a week of 5 
minutes each, are not sufficient to produce a measurable difference. 

Beech et al. [1970] also investigated extensively the attitudes 
of parents and students to the program. The results are of some 
significance for two reasons. The terminals were located in the homes 
and not in the school, and the children were in all cases drawn from 
poverty areas. A survey of the attitudes of the parents toward this 
kind of program showed generally positive attitudes. The results of 
a questionnaire directed to the students also indicated a favorable 
response. While positive attltudinal responses to this experiment 
must be interpreted as preliminary, they do suggest that further 
research on bringing instruction into the home via telephone is worth 
iuvestigatlng. 

In another study relating to attitudes Smith and Hess [1972] 
examined no:- -cognitive effects of CAI in their research. The measures 
of student attitudes included the Sears Self-Concept Inventory, 
Coopersmith Self-Esteem Inventory, Crandall Locus of Control Instrument 
and items from the Coleman report. All of these measurement instru- 
ments are based on student resronses and question the student's 
attitudw relfctlng to general control over envlronnent, responsibility 
for mathematics failures and successes, aptitude in mathematics and 
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social relations. The sample used consisted of 159 students with CAI 
and 161 without in grades 7. 8 and 9. The students were using the 
mathematics strands drill-and-practice program. The general result 
was no difference in means between CAI and non-CAI groups and no 
difference in means for pre-test and poet-test measurements of attitudes 
for the CAI group. 

Two studies have related arithmetic achievement to amount of 
CAI. using regression models of the sort described in Section I. Wells. 
Whelchel. and Jamison [to appear] analyzed data for 446 fifth- and 
sixth-grade students. The analysis was done on an individual student 
basis and it was possible to match students with their teachers. Data 
were separated by grades of students and then stratified by sex on the 
assumption that a differential effect of school resource variables might 
be observed. 

The ^'ependent variable in the regression model was the score of 
the student on the mathematics portion of the California Test of Basic 
Skills at the end of the experimental year (MA). Independent variables 
Included test score at the beginning of the year (MB), years of teacher 
experience (TEAEXP). score of teacher on a 100-point verbal test 
(TEAVER) . teacher degree level (TEADEG) . student self-efficacy (SELFEX) . 
and the number of sessions of CAI for each student during the course 
of the year (CAISFS) . Students were not assigned a number of CAI 
sessions randomly and. with the exception of sixth grade boys, there was 
a slight positive correlation between MB and CAISES. The test scores 
were measured in grade equivalents. MulticolUnearity was a problem 
only with the teacher characteristic variables and separate models were 
specif jBd for each of the teacher variables. Both linear and Cobb-Douglas 
(log-log) models wera tested. The equation reported below for fifth- 
grade males with CAI la representative of the results obtained for the 
various stratifications of students. The t values are In parentheses. 

Pifth-grcde males with CAI. Cobb-Douglas m^del (all variables are in logs): 

MA - .1408 + .8052 MB + .0572 CAISES - .0643 SELFEX + .0195 TEAEXP 
(13.08) (3.60) (1.61) (1.81) 

- .7427. 



The Cobb-Douglaa model measures output elasticity; a 1% increase 
in the number of CAI sessions, for example, would result In a 5.72% 
increase in mathematics achievement. For this model the gain from 100 
sessions would depend on the initial level of achievement. Among 
students who had some CAI In this sample the average number of CAI 
sessions ranged from 59.25 for sixth grade girls (with a standard 
deviation of 36.44) to 85.00 for fifth grade girls (with a standard 
deviation of 37.68). The aunber of CAI sessions were statistically 
significant In both models for fifth-grade males and the Cobb-Douglas 
models for fifth-grade females; they were statistically Insignificant 
in the linear models for fifth grade-females and in both models for 
sixth-grade students. 

Slopes, Fletcher, Zanotti, Lorton, and Searle [to appear] reported 
a 1971-72 study dealing with the effects of drill and practice in 
elementary mathematics on elementary-school children in residential 
schools or day classes for 312 deaf students. The number of sessions 
students were to receive in a 5-4nonth period was randomly assigned, 
eliminating multicollinearity problems. A number of different models 
in addition to the linear regression model were tested. Application of 
the linear model yielded the following regression equation: 

E(T^2> ■ 1-116 + .793 T^^ + .084 , 

^^^^ ^11 ^ pretest score of student 1 on a modified Stanford 
Achievement Test, T^^ is the posttest score on a second form of the 
same test, and la the nuiber of CAI sessions of student 1 divided 
by 10. The multiple correlation obtained was .811. It should be noted 
that if linearity held In 150 CAI sessions, a gain of 15 X .084 - 1.26 
grade-placement years would be expected. In fact, in the experiment the 
average number of sessions of the groi;^ with the most intensive treatment 
was 75.84 with a standard deviation of 29.15, and linearity held over 
this range reasonably well. 

An extensive analysis of detailed student learning and performance 
data on elementary^ chool CAI mathematics may be found In a recent 



- 43 - 



book by Suppes and Mornlngstar [1972]. The results are too extensive 
to survey here. Much of the volune Is devoted to analysis of the sort 
of mlcromodels we have excluded from consideration In this review. 

In the case of beginning reading, a ntniber of CAI studies have 
been reported by Atkinson and his collaborators. Results of a tutorial 
reading program in 1966-67 in which students were given approximately 20 
minutes a day on teminals are reported in Atkinson [1958]. The results 
of this experiment are Interesting, becaute while the experimental group 
received tutorial reading via CAI the control group in this study 
received tutorial mathematics via CAI; therefore, both groiqis were 
being exposed to CAI. The experimental and control groups had similar 
characteristics; they constituted the approximately 100 students in the 
first grade in the school in which the experiment was conducted 
(approximately 100 because the ramber enrolled varied slightly during 
the school year). The posttest results for the experimental and control 
groups on the California Achievement Test and the Hartley Reading Test 
are shown In Table 7. As can be seen, most of the results are favorable 



Insert Table 7 about here 



to the experimental group. The computer systoa used in this experiment 
was an expensive one, for the student stations had not only a cathode- 
ray tube and keyboard terminal, but also an audio and visual display 
unit as well as a ligjit pen for the cathode-ray tube. 

Evaltiatlon of a recent CAI program in Initial reading using only 
teletype terminals and audio (but computer-generated audio) Is reported 
In Fletcher and Atkinson [1972] . The curriculum was conceived as 
supplementary drlll-and-practlce rather than tutorial. The efficacy of 
the program was tested by using a gcovp of 50 mat died pairs of students. 
Prior to receiving exposure to CAI, 25 pairs of first-grade boys and 25 
pairs of first-grade girls were matched on the basis of the Metropolitan 
Readiness Test, which was adilnlstered in November 1969. Three posttests 
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Posttest Resvdts for Experimental and Control Groups 
(Atkinson [I968]) 



Test 


Experimental 
group 


Control 


p value 


California Achievement litest 








Vocabulary 


^5.91 


36.10 


< .01 


Cauprehension 




40.62 


- 


Total 


45.63 


39.61 


< .01 


Hartley Reading Test 








Form class 


11.22 


9.00 


< .05 


Vocabulary 


19.38 


17.05 


< .01 


Phonetic discrimination 


30.88 


25.15 


< .01 


Pronunciation 








Nonsense word 


6.03 


2.30 


< .01 


Word 


9.95 


5.95 


< .01 


Recognition 








Honserise word 


18.45 


15.25 


< .01 


Word 


?'-.6l 


16.60 


< .01 



were administered in late May and early June 1970. Four subtests of 
the Stanford Achievement Teat were used. The California Cooperative 
Primary Reading Test, as well as a special test developed at Stanford, 
were also administered. The average grade placement on the Stanford 
Achievement Test and on the California Cooperative Primary Test, which 
were used as posttests. is shown in Table 8. While the results are 
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significant in favor of the CAI groups, what is especially interesting 
is the unusually good perfomance of the boys. Similar results where 
boys did about as well as girls in a CAI reading environment were also 
reported In Atkinson [1968]. These results are contrary to those 
ordinarily obtained In TI for initial reading performance of boys 
and girls. 



C ollege Level CAI 

A variety of evaluations have been conducted at the college 
level, mainly in connection with courses operated as part of research 
and development projects in CAI. Although it is not possible to give 
a complete summary here, major efforts made at a nuni>er of institutions 
are summarized and include studies conducted at Florida State University, 
the State University of New York at Stony Brook. University of Illinois. 
University of Texas, and Stanford University. 

Hansen. Dick, and Lippert [1968] of Florida State University 
reported results of implementing collegiate Instruction in physics by 
means of CAI. that la. problflm sessions were handled in a CAI environment. 
In the fall of 1967 three groups of students were compared: (a) students 
receiving the bulk ^f Instruction by CAI. (b) students receiving partial 
CAI and partial TI. and (c) students receiving only TI. Correlated t 
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Average Grade Placement on the Stanford Achievement Test (SAT) 
and the California Cooperative Primary Test (OOOP) 
[Fletcher & Atkinson, 1972] 





£AT 


COOP 




GAI 


2.2 


2.5 












non-CAI 


1.8 


1.8 




GAI 


2.k 


2.6 


Girls 










non-CAI 


2.0 


2.2 
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tests on the sum of the Mdtem and final examination scores showed that 
the autonomous CAI group was statistically superior to the other groups, 
but the difference between students who received partial CAI and students 
who received only TI was not significant. 

In the spring of 1970 three more groups of students were studied. 
One vas an autonomous CAI group, a second was a group of students re- 
ceiving TI only, and a third vas a group of students receiving TI plus 
a 4-hour-examlnation review on a computer system. The mean scores for 
the midterm examination, the final examination, and final grade showed 
xio significant differences among the three treatment groups. The effect 
of CAI seemed to truncate the distribution of lower grades. The 
investigators applied a Koljnogorov-Smlmov test to the distribution of 
scores from the CAI group and the TI group receiving the review on the 
computer system. The results of this test Indicated that the two distri- 
butions were significantly different (D - 8.48; p < .05). with the 
CAI group receiving fewer low grades than the TI group. 

The CAI Center at Florida state Unlveisity has also conducted 
several studies on ccmputer-managed instruction (CMI) . CMI differs from 
CAI In that students do not Interact on line with the computer system, 
but rather they receive from the computer program directions of what unit 
to do next, possibly diagnostic testing, and remedial Infomation. If 
necessary. Hagerty [1970] reported the results of a CMI course In tech- 
niques of PI conducted In the fall of 1969 with 59 graduate students. 
Students worked at their own pace by scheduling tine on the computer 
temlnal as needed. The results Indicated that the CMI students performed 
as well as students taugjht the previous year by conventional classroom 
lecture methods. What Is Interesting about this study. whi.A did not 
produce a significant difference In the two groups, vas that the costs 
of operating the course for terminal tine and personnel were $3,074^ 
which Is lower than the costs for TI. 

Lawler [1971] Investigated the differential effects of instruc- 
tional strategies In CMI. using 167 undergraduates in a health education 
course at Florida State University. Forty-one of the students received 
TI; the remaining students were randomly assigned to one of three CMI 
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treatments. The three CMI treatment groups were varied In their pace 
through the course. The results showed superiority of the CMI groups 
over the TI group on final examination perf onnances . Again, the results 
of the extensive analysis of variance are too detailed to suimarize h. a. 
but the general conclusion Just stated is supported by extensive 
statistical analysis. Concerning the different CMI treatments, there 
seems to be some advantage to requiring students to reach mastery at 
each stage or level of the course.^® 

Adams [1969] and Morrison and Adams [1969] described results of 
experiments conducted over two years at the State University of New York, 
Stony Brook. The subjects were students In introductory German, and 
both CAI and control groups received 3 hours of Instruction per week 
In regular classes. The control group received, in addition to class 
time, the standard 1 hour per week of language laboratory; the CAI group 
received instead 1 hour per week of CAI in reading and writing. At very 
slight (if any) sacrifice to their performance In listening and speaking, 
the CAI students performed substantially better than the control students 
on tests of reading and writing achievement. The CAI and control groups 
were well matched on the Modem Language Aptitude Test. The exper- 
imenters reported a generally favorable student attitude to CAI. 

Using the PLATO system at the University of Illinois, Grandey 
[in press] studied the use of computers to aid instruction in beginning 
chemistry. Thirty-one students with weak high sdiool chemistry back- 
grounds were exposed to varying amounts of material presented by PLATO. 
Coaqparisons were made between 18 students who used PLATO for instruc- 
tion and 13 students who used PLATO only for review. No significant 
differences between the two groups were established. Axeen [1967] 
studied CAI in the use of the library by undergraduates. Cooparing 32 
students who received a sequence of PLATO lessons, which entirely replaced 
an Introductoxy college course on library t^e, with 34 students taught ty 



Saker [1971] reviewed five additional CMI projects but reported 
no data on the Instructional effectiveness of any of them. Kelley 
[1972], In a later paper, did present detailed results on the effectiveness 
of one of the projects described by Baker. 
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ordinary TI. Axeen found that students In the experimental class took 
less tl»e to learn the same material. However, no significant differences 
were noted In performance as measured by the Library Orientation Test 
for college freshmen. 

Bltzer and Boudreaux [1969] used the PLATO system fcr a CAI 
course In nursing. One hundred forty-four nursing students spent an 
average of 20 hours In a course on maternity nursing and 18 hours In 
a course on pharmacology for nurses. Perhaps the most striking result 
obtained In this study was the savings, in time. All members of one 
group of 38 nurses for which complete time measures were kept finished 
the maternity nursing material In 50 hours or less at terminals. The 
same material required 84 hours of standard lecture presentation. 

Coottb-'j and Peters [1971] used the PLATO system to study CAI 
in role-playing games. One hundred six students in an Introductory 
American government course spent 18 class hours at terminals. Comparisons 
were made with a Uke number of students who received TI during 18 hours 
in small-group discussions. No significant differences in the two groups 
were obtained. 

At the University of Texaa a nunber of experiments and demonstra- 
tion r Jects have been completed in the Computer Assisted Instruction 
Laboratory. Castleberry and Lagowskl [1970] reported on a CAI chemistry 
course. Fifteen CAI modules were developed ss supplementary material 
for tbft introductory courae in general chemistry. The following results 
were obtained. In both semesters during the academic year 1968-69, 
students who took sdvantage of the available CAI modules scored signif- 
icantly higher than the control group on the parts of the final exam- 
ination covered by the modules. In addition, during the first semester, 
students using the CAI modules also scored significantly higher than 
the control group on the parts of the final examination not covered by 



19 

The PLATO system is currently being expanded and a detailed 
r"Tu*'™-^ expanded system, as well as a syst« being constructed 
by the MTTBE Corporation, wiU be conducted over the next four years. 
Anastaslo [1972] described the plans for the evaluation and Lyman [1972] 
provided a listing of previous PLATO research and curriculum efforts. 
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the Bodules. These ti,o results alone suggest that a selective process 
was at work rather than any Instructional advantages of the CAI modulee. 
since students used the CAI modules on a voluntary basis. However, during 
the second semester, no significant differences were observed between 
the experimental and control groups on the final examination In the 
material not covered by the nodules. The results taken together support 
the hypothesis that the CAI modules were a useful addition to the course. 
Evidence is presented In Table 9 that the voluntary experimental group 



Insert Table 9 about here 



using the CAI modules was not necessarily more able than the control 
group. For example, the Stanford Achievement Test scores In both 
semesters were no better for the experimental group than for the control 
group; If anything, they were perhaps sUghtly lower. The same Is true 
of the results on the chemistry placement score. 

Judd, Bunderson, and Bessent [1970] Investigated the effects of 
learner control In a CAI course in precalculus mathematics. They did 
not compare the student performance with TI classes, but they did conpare 
learner-controlled strategies wltli program-controlled strategies. No 
striking differences were found. One general conclusion did emcirge from 
their analysis. Student control of progress through a course seems to 
be successful In subjects In which the student has competence and 
IB definitely less successful when the student's competence Is low, 
or he has Uttle familiarity with the course material on the basis of 
past experience. This study Illustrates how difficult It Is to obtain 
strong conclusions about how leaner control should be built Into CAI 
courses. As In other areas of researdi on the effectiveness of 
Instructional methods, interaction between the cognitive and affective 
states of the student and the structure of Instruction will certainly 
be a major focus of Investigations In the next few years. 



Group Means [Castlelserry & Lagovski, 1970] 



Variable 


Experimental 
group 


Control 
group 


Experimental 
dropouts 


First Semester 








Final Exaa Score: 








Items related to 
modides 


86.7 


68.7 


71^.9 


Final Exam Score: 








Items not related 
to module 


83.5 


7l*.l 


75.7 


S/IT-Math 


51fl 


530 


1^80 


SAT-Verb&l 


572 


5l^5 




Chemistry placement 
scQre®" 


19.0 


19.7 


17.2 


second oaneBTer 








Final Exaa Score: 








Items related to 
modvtles 


81.3 


71.8 


76.8 


Final Exam Score: 








Items not related 
to modiGLes 


U2.6 


k2.S 


k2.6 


S^T-Math 


1*80 


515 


518 


aiT-Verbal 


517 


557 


537 \ 


Chemistry placement 
score* 


15.5 


£;j.7 


I6.k 



^^Ibe Chanistry Placement Qxemlnation is required of all stuOents 
before they register in general chemistry. The maximum score on this 
examination is 50. 
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Hoaeyer [1970] reported the results of comparing a Qil with a TI 
course In computer programming. The students taking either the CAI 
version or the lecture version were required to have had at least one 
previous course In computer science and some experience In programming. 
Two sections of a course were used: one as a CAI group and the other 
as a lecture group. There were ten students in each group. Although 
the number of students was small, the study explicitly tested the 
following h3rpothe8es: 

HI. The CAI group can complete course instruction significantly 
faster than the lecture group. This hypothesis was accepted; the CAI 
group completed course Instruction about twice as fast in tenns of nuid>er 
of hours (an average of 13.75 hours for the CAI group compared with 24 
hours for the lecture group). 

H2. The CAI group makes significantly fewer personal visits 
to the instructor. This hypothesis was rejected. Both groups made about 
the same nuaber of personal visits to the office of the instructor. 

H3. There is no significant difference between the CAI and 
lecture groups with respect to mean scores on examinations. This 
hypothesis was accepted. The performance of the students was not 
significantly different at the .05 level. 

H4. There is no significant difference between CAI and 
lecture groups with respect to mean grades on computer prograw written. 
This hypothesis also was accepted, with about equal performance from 
the two groups. 

Edwards and Judd [1972] reported on the evaluation of a course in 
special education for undergraduates at the University of Texas. 
Students in the course were assigned to one of three groups. One group 
received a course handbook and participated in a discussion section; 
the second group Joined only a discussion section; and a third gtovp 
received the handbook and CAI, but did not participate in a discussion 
section. The results were soawmfaat adzed, but the evidence favored 
tiie test performance of the group receiving CAI rather than discussion 
sections, irtilch Indicates that In this kind of course, CAI can 
successfully replace small group sections of Urge lecture courses. 
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At Stanford University, Joseph Van Caapen has developed a full 
two-year tutorial course In Introductory Russian. This project, begun 
In 1967, teaches the standard aspects of a first-year course at the 
college level, that is to say, comprehension of written Russian, com- 
pr^ension of spoken Russian, and mastery of grammar and syntax. Of 
the three main conponents cf a college-level language course — regular 
classroom sessions on a daily basis, time spent in the language 
laboratory, and regular hcmevork assignments — only the functions of 
the tutorial classroom sessions are assumed by the CAI course. In 
addition to their time at computer consoles, students spend time in the 
language laboratory and do off-line homework assignments. What is 
important about this example is that the regular 5 hours a week of 
classroom instruction were completely replaced by dally work for a 
comparable time at cooiputer texminals. 

An evaluation of the course for 1968-69 Is presented In Suppes 
and Momlngstar [1969]. First of all, the CAI course shoved superior 
holding power in comparison with TI. Of the 30 students originally 
enrolled In the CAI course, 73 percent finished all three quarters of 
the first year, ^reas of the 38 students in the two r^ular classes 
only 32 percent finished the year's curriculum. Approximately 66 percent 
of the content of the final cxodLnations for the autuan and winter 
quarters were identical for the CAI and regular Russian courses; the 
final examination for the spring quarter was identical for the two 
groups. The average number of errors was lower for the CAI studtAits 
in all three quarters and was statistically significant for the fall 
quarter (Mann-Whltney D test, p < .001) and the spring quarter 
(p < .05) , but not for the winter quarter. Since the selection process 
resiilting from nore of the poorer students' leaving the regular course 
biased results against the CAI group, the superiority of the CAI group 
on the spring examination la more Impressive than the statistical 
analysis Indicates. 
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Conclusion 



As In other methods of Instruction surveyed In this report, no 
simple uniform conclusions can be drawn about the effectiveness of CAI. 
At the elementary-school level, CAI is apparently effective as a 
supplement to regular Instruction. What we do not have are the sorts 
of experiments required for a complete productivity analysis. There are 
no examples yet of CAI's being Introduced with a concomitant change in 
student-teacher ratio, which would, for example, cover the costs of CAI. 
At the present time, we can only conclude that CAI can be used In some 
situations to Improve achievement scores, particularly for disadvantaged 
students. 

At the secondary school and college levels, a conservative con- 
clusion Is that CAI Is about as effective as TI when It Is used as a 
replacement. It may also result In substantial savings of student 
time In some cases. Since the equal-effectiveness conclusion seems to 
be broadly correct for wst alternative methods of Instruction at the 
college level, there should be in the future increasing opportunities 
to experiment with selecting the method of instruction in terms of costs, 
and real opportunities should exist for substituting capital for labor, 
especially as the relative costs of technology In comparison to labor 
decline over titm next decade. 



VII. CONaUSION 



In this paper «e have surveyed research on the effectiveness 
of traditional instruction (TI) . instructional radio (IR) . instructional 
television (ITV) . prograaned instruction (PI), and computer-assisted 
Instruction (CAI). Students learn effectively from ail these media, 
and relatively fev studies Indicate a significant difference in one 
nedltiD over another or of one variant of a medium over another. The 
studies taken together suggest that alternative methods of TI are 
approximately equally effective, although several studies indicated that 
different variables are significantly correlated with student achievement. 
Teacher verbal ability was Important In many of these studies, and the 
evidence suggests that smaller class size may consistently, if sUghtly. 
Improve the performance of primary-grade students. 

Though there Is a substantial past Mjg^jg^lyln the use of IR. 
few studies of its effectiveness exist- , - ^maber that do exist were, 
however, carefully done and they indicate that IR. supploaented with 
appropriate printed material. Is about as effective as TI. There l^^A^^-* 
much more extensive research literature on the effectiveneas o^^i^'" 
and excellent surveys of that literature already exist. ^^^\s strong 
evidence that ITV. used In a way that closely simulat^^l. la as 
effective, on the average, as TI for all grade le^ii and subject matters. 
There is very little evidence concerning the effectiveness of ITV used 
In ways that utiUze the unique capabiUties .f the medium. A reasonable 
fraction of the student and teacher populaf^ons has a somewhat unfavorable 
attitude toward ITV. although the Incidence of unfavorable attitudes 
tends to diminish as Institutions gain experience with the medlim. After 
auch experlance a majority of students h&r4 neutral or favorable 
attitudes toward ITV. 

Both PI and CAI attempt to Improve the quality of Instruction 
by providing for its lndlvl>dallzation along one or more dimensions. 
Nonetheless, findings of ""fxi significant difference" domlnata the 
research literature in thi area. Though there are often no significant 

/ 
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differences in achievement some of the studies do report a saving in 
student time, and this la an index of success. When small amounts of 
CAI are used as a supplement to regular classroom instruction (as with 
the elementary-school drill-and-practice programs) substantial evidence 
suggests that it leads to an Improvement in achievaaent, particularly 
for slower students. Models exist that relate the amount of achievement 
gain to the number of CAI sessions a student receives. 

In broad terms, the many studies we have surveyed suggest that 
the costs of alternative technologies, with capital investment amortized 
over an appropriate nunber of years, should always be given serious 
consideration In planning an educational program or evaluating proposed 
changes iu current programs. On the other hand, there are enough 
differences in the studies in texms of achievement measures to suggest 
^tij^r^«i*^licy of strict minimization of costs in the choice of a 
.^yi^'^^^^^^osr fox *taching is too alnple a criterion. At least four 
considerations will probably be of Importance in the future. Each 
will need more extensive study: 

First, we must examine if the savings in time exhibited in 
some of the studies using PI or CM can be shonn to be significant 
over longer periods and for a higher percentage of the total instructional 
program of students. 

Second, we do not yet have an appropriately detailed evaluation 
of the Impact of the various technologies on the long-term motivation 
of students. 

Third, the long-term effects of individualization and privacy of 
learning characteristic of sane of the technologies also needs more 
extensive evaluation. We do not know, for example, whether students 
who are given highly Individualized programs in the elanentary school 
for most of their instruction will strongly prefer the continuation of 
such methods in secondary school and college or whether they will desire 
to return as they grow older to more traditional foxms of instruction. 

Fourth, it has been indicated at a nuaber of points in this 
review that most evaluations, particularly those considered well con- 
trolled, compare TI to a fom of IR, ITV, or CAI that closely enulates 



the TI. It is at least plausible that many of the conclusions of this 
survey would be overturned were aore Imaginative uses of the nedla 
explored, that yet permitted comparative evaluation. 

Most of the educational technologies we have surveyed in this 
article have a relatively recent history. Even though there is already 
a fairly extensive literature on their evaluation, it would be a mistake 
to view the present state of that literature as anything but preliminary 
In nature. It will be many years before we have an adequately deep 
assessment of the strengths and weaknesses of the technological alter- 
natives to traditional Instructioa that have been considered In this 
survey. 
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